Methods of Statistical Data Compression

Data compression is important not only for conserving resources; it also has applications in cryptography and it can be used as an estimator for redundancy in the data: this has many applications, such as prediction, classification and other difficult problems in machine learning. We study algorithms that perform lossless statistical data compression. Statistical data compression is attractive because it allows for separation of the problems of modelling and coding, both of which will be treated here. It seems safe to say that with the development of arithmetic coding in 1976, the problem of coding has been solved satisfactorily, while the problem of modelling remains very difficult to this day. We will restrict ourselves to online modelling. In chapter 2 we study the theoretical background of statistical data compression, relating results of information theory and probability theory to coding and modelling. Then we focus on more concrete issues: in chapter 3 we treat an adaptation of Ukkonen’s algorithm for the online construction of suffix trees, which we use to maintain the particular information about the input stream that we need for the construction of the kind of models that we study. We will find that, for our purposes, a problem with online suffix trees is that they do not explicitly represent the end of the input stream in the tree structure. An investigation of the subtle properties of online suffix trees will help us to formulate a solution to this problem. Finally, in chapter 4 we describe PPM and point out its inherent suboptimality. We then develop two alternative source models which are motivated by entirely different arguments. A comparison will show that these alternatives provide promising performance, warranting further research.